Allow more than one http rest & grpc listeners #3749
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dkalinowski
changed the title
dkalinowski
changed the title
dkalinowski
commented
Nov 6, 2025
dkalinowski
commented
Nov 6, 2025
dkalinowski
commented
Nov 7, 2025
dtrawins
reviewed
Nov 7, 2025
docs/performance_tuning.md
Outdated
|
|
||
| ## Network Configuration for Optimal Performance | ||
|
|
||
| When clients connect to the server using hostname resolution (particularly "localhost"), the system may attempt IPv6 resolution first before falling back to IPv4. If IPv6 is disabled, misconfigured, or unavailable, this can cause connection timeouts and delays before the IPv4 fallback occurs, which is especially noticeable when minimizing time to first token in generative AI applications. |
Collaborator
There was a problem hiding this comment.
By default, OVMS endpoints are bound to all ipv4 addresses. On same systems, which route localhost name to ipv6 address, it might cause extra time on the client side to switch to ipv4. It can effectively results with extra 1-2s latency.
It can be overcome by switching the API URL to http://127.0.0.1 instead.
Alternatively ipv6 can be enabled in the model server using --grpc_bind_address and --rest_bind_address.
For example:
--grpc_bind_address 127.0.0.1,::1 --rest_bind_address 127.0.0.1,::1
or
--grpc_bind_address 0.0.0.0,:: --rest_bind_address 0.0.0.0,::
dtrawins
approved these changes
Nov 7, 2025
atobiszei
approved these changes
Nov 12, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🛠 Summary
CVS-170537